Inline categorizing of events

ABSTRACT

Embodiments are directed to managing operations over a network. An event associated with network operations may be provided. A hash function may generate key values that correspond to words included in message information associated with the event. A message vector may be generated based on the key values such that each component in the message vector that corresponds to a key value is set to one. Group vectors may be determined such that each group vector is associated with an event group. Similarity scores may be generated for the group vectors based on the message vector and the group vectors such that each group vector corresponds to a separate similarity score. If the similarity scores exceed a threshold, the event may be associated with event groups associated with a group vector that correspond similarity score that exceeds the threshold.

TECHNICAL FIELD

The present invention relates generally to computer operations and moreparticularly, but not exclusively to managing events associated withcomputer operations.

BACKGROUND

IT systems are increasingly becoming complex, multivariate, and in somecases non-intuitive systems with varying degrees of nonlinearity. Thesecomplex IT systems may be difficult to model or accurately understand.Various monitoring systems may be arrayed to provide alerts,notifications, or the like, in an effort to provide visibility tooperational metrics, failures, and/or correctness. However, the sheersize and complexity of these IT systems may result in a flooding ofdisparate event messages from disparate monitoring/reporting services.Today with the increased complexity of distributed computing systemsevent reporting and/or management may overwhelm IT teams tasked tomanage them. At enterprise scale, IT systems may have millions ofcomponents resulting in a complex inter-related set of monitoringsystems that report millions of events from disparate subsystems. Manualtechniques and pre-programmed rules are labor intensive and expensive,especially in the context of large centralized IT Operations with verycomplex systems distributed across large numbers of components. Further,these manual techniques may limit the ability to scale and evolve forfuture advances in IT systems capabilities. Thus, it is with respect tothese considerations and others that the present invention has beenmade.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive embodiments of the present innovationsare described with reference to the following drawings. In the drawings,like reference numerals refer to like parts throughout the variousfigures unless otherwise specified. For a better understanding of thedescribed innovations, reference will be made to the following DetailedDescription of Various Embodiments, which is to be read in associationwith the accompanying drawings, wherein:

FIG. 1 illustrates a system environment in which various embodiments maybe implemented;

FIG. 2 illustrates a schematic embodiment of a client computer;

FIG. 3 illustrates a schematic embodiment of a network computer;

FIG. 4 illustrates a logical architecture of a system for inlinecategorizing of events in accordance with at least one of the variousembodiments;

FIG. 5 illustrates a logical flow of a portion of the operations of aclustering engine for inline categorizing of events in accordance withone or more of the various embodiments;

FIG. 6 illustrates a logical flow of a portion of the operations of aclustering engine for inline categorizing of events in accordance withone or more of the various embodiments;

FIG. 7A illustrates the logical flow for generating or adapting learnerobjects in accordance with one or more of the various embodiments;

FIG. 7B illustrates the logical flow for employing learner objects inaccordance with one or more of the various embodiments;

FIG. 8 illustrates an overview flowchart for a process for inlinecategorizing of events in accordance with one or more of the variousembodiments;

FIG. 9 illustrates a flowchart for a process for inline categorizing ofevents in accordance with one or more of the various embodiments; and

FIG. 10 illustrates a flowchart for process 1000 for inline categorizingof events in accordance with one or more of the various embodiments.

DETAILED DESCRIPTION OF VARIOUS EMBODIMENTS

Various embodiments now will be described more fully hereinafter withreference to the accompanying drawings, which form a part hereof, andwhich show, by way of illustration, specific exemplary embodiments bywhich the invention may be practiced. The embodiments may, however, beembodied in many different forms and should not be construed as limitedto the embodiments set forth herein; rather, these embodiments areprovided so that this disclosure will be thorough and complete, and willfully convey the scope of the embodiments to those skilled in the art.Among other things, the various embodiments may be methods, systems,media or devices. Accordingly, the various embodiments may take the formof an entirely hardware embodiment, an entirely software embodiment oran embodiment combining software and hardware aspects. The followingdetailed description is, therefore, not to be taken in a limiting sense.

Throughout the specification and claims, the following terms take themeanings explicitly associated herein, unless the context clearlydictates otherwise. The phrase “in one embodiment” as used herein doesnot necessarily refer to the same embodiment, though it may.Furthermore, the phrase “in another embodiment” as used herein does notnecessarily refer to a different embodiment, although it may. Thus, asdescribed below, various embodiments may be readily combined, withoutdeparting from the scope or spirit of the invention.

In addition, as used herein, the term “or” is an inclusive “or”operator, and is equivalent to the term “and/or,” unless the contextclearly dictates otherwise. The term “based on” is not exclusive andallows for being based on additional factors not described, unless thecontext clearly dictates otherwise. In addition, throughout thespecification, the meaning of “a,” “an,” and “the” include pluralreferences. The meaning of “in” includes “in” and “on.”

For example embodiments, the following terms are also used hereinaccording to the corresponding meaning, unless the context clearlydictates otherwise.

As used herein the term, “engine” refers to logic embodied in hardwareor software instructions, which can be written in a programminglanguage, such as C, C++, Objective-C, COBOL, Java™, PHP, Perl,JavaScript, Ruby, VBScript, Microsoft .NET™ languages such as C#, or thelike. An engine may be compiled into executable programs or written ininterpreted programming languages. Software engines may be callable fromother engines or from themselves. Engines described herein refer to oneor more logical modules that can be merged with other engines orapplications, or can be divided into sub-engines. The engines can bestored in non-transitory computer-readable medium or computer storagedevices and be stored on and executed by one or more general purposecomputers, thus creating a special purpose computer configured toprovide the engine.

The term “organization” as used herein refers to a business, a company,an association, an enterprise, a confederation, or the like.

The term “operations management system” as used herein is computersystem that may be arranged to monitor, manage, and compare, theoperations of one or more organizations. Operations management systemmay be arranged to accept various Operations events that indicate eventsand/or incidents occurring in the managed organizations. Operationsmanagement systems may be arranged to manage several separateorganizations at the same time. These separate organizations may beconsidered a community of organizations.

The terms “event” as used herein refer one or more data structures ormessages the may report outcomes, conditions, or occurrences that may bedetected or observed by an operations management system. Operationsmanagement systems may be configured to monitor various types of eventsdepending on needs of an industry and/or technology area. For example,information technology services may generate events in response to oneor more conditions, such as, computers going offline, memoryoverutilization, CPU overutilization, storage quotas being met orexceeded, applications failing or otherwise becoming unavailable,networking problems (e.g., latency, excess traffic, unexpected lack oftraffic, intrusion attempts, or the like), electrical problems (e.g.,power outages, voltage fluctuations, or the like), customer servicerequests, or the like, or combination thereof.

Events may be provided to the operations management system using one ormore messages, emails, telephone calls, library function calls,application programming interface (API) calls, including, any signalsprovided to an operations management system indicating that an event hasoccurred. One or more third party and/or external systems may beconfigured to generate event messages that are provided to theoperations management system.

The term “incidents” as used herein may refer to a condition or state inthe managed networking environments that requires some form ofresolution by a user or automated service. Typically, incidents may be afailure or error that occurs in the operation of a managed networkand/or computing environment. One or more events may be associated withone or more incidents. However, not all events are associated withincidents.

As used herein the term “configuration information” refers toinformation that may include rule based policies, pattern matching,scripts (e.g., computer readable instructions), or the like, that may beprovided from various sources, including, configuration files,databases, user input, built-in defaults, or the like, or combinationthereof.

The following briefly describes embodiments of the invention in order toprovide a basic understanding of some aspects of the invention. Thisbrief description is not intended as an extensive overview. It is notintended to identify key or critical elements, or to delineate orotherwise narrow the scope. Its purpose is merely to present someconcepts in a simplified form as a prelude to the more detaileddescription that is presented later.

Briefly stated, various embodiments are directed to managing operationsover a network. In one or more of the various embodiments, an event thatmay be associated with one or more operations in the network may beprovided.

In one or more of the various embodiments, a hash function may beemployed to generate one or more key values that correspond to one ormore words included in message information that is associated with theevent.

In one or more of the various embodiments, a message vector may begenerated based on the one or more key values such that each componentin the message vector that corresponds to a key value may be set to avalue of one.

In one or more of the various embodiments, one or more group vectorsthat have a same number of components as the message vector may bedetermined such that each group vector is associated with an eventgroup.

In one or more of the various embodiments, one or more similarity scoresmay be generated for the one or more group vectors based on the messagevector and the one or more group vectors such that each group vectorcorresponds to a separate similarity score. In some embodiments,generating the one or more similarity scores may include computing oneor more cosine similarity values based on the message vector and each ofthe one or more group vectors such that the one or more cosinesimilarity values may be employed as the value of the one or moresimilarity scores.

In one or more of the various embodiments, in response to a portion ofthe one or more similarity scores exceeding a threshold value, the eventmay be associated with one or more event groups such that each eventgroup may be associated with a group vector that that corresponds to theseparate similarity score that exceeds the threshold value. In someembodiments, associating the event with the one or more event groups mayinclude adding the message vector to each group vector that may beassociated with the one or more event groups.

In one or more of the various embodiments, a learner object may bedetermined based on an association of the learner object with one ormore of a user, an account, a service, or an organization. In someembodiments, the learner object may be employed to generate one or moreagreement scores based on the message vector and the one or more groupvectors such that each group vector corresponds to a separate agreementscore. In some embodiments, in response to a portion of the one or moreagreement scores exceeding an agreement threshold value, the event maybe associated with one or more event groups such that each event groupmay be associated with the group vector that corresponds to the separateagreement score that exceeds the agreement threshold value. And, in someembodiments, in response to another portion of the one or more agreementscores being less than a disagreement threshold value, the event may bedisassociated from each of the one or more event groups that may beassociated with a group vector that may be associated with the otherportion of agreement scores.

In one or more of the various embodiments, the hash function may beemployed to generate one or more additional key values that correspondto one or more pairs of words that may be included in the messageinformation. And, in some embodiments, the one or more additional keyvalues may be included in the message vector such that each additionalkey value may correspond to a component in the message vector thatcorresponds to another key value may be set to a value of one.

In one or more of the various embodiments, non-semantic information maybe determined from the message information based on one or more ofpattern matching, parsing, regular expressions, or the like. In someembodiments, the non-semantic information may be removed from themessage information such that the non-semantic information may beexcluded from the message vector.

In one or more of the various embodiments, feedback information may beprovided from a user such that the feedback may be one or more or moreof associating another event with an event group or disassociating theother event with the event group. In one or more of the variousembodiments, a learner object may be generated based on another messagevector that may be associated with the other event and a group vectorthat may be associated with the event group. And, in some embodiments,the learner object may be employed to generate agreement scores that maybe associated with the user.

Illustrated Operating Environment

FIG. 1 shows components of one embodiment of an environment in whichembodiments of the invention may be practiced. Not all of the componentsmay be required to practice the invention, and variations in thearrangement and type of the components may be made without departingfrom the spirit or scope of the invention. As shown, system 100 of FIG.1 includes local area networks (LANs)/wide area networks(WANs)-(network) 111, wireless network 110, client computers 101-104,monitoring server computer 114, operations management server computer116, application server computer 118, or the like.

At least one embodiment of client computers 101-104 is described in moredetail below in conjunction with FIG. 2. In one embodiment, at leastsome of client computers 101-104 may operate over one or more wired orwireless networks, such as networks 110, or 111. Generally, clientcomputers 101-104 may include virtually any computer capable ofcommunicating over a network to send and receive information, performvarious online activities, offline actions, or the like. In oneembodiment, one or more of client computers 101-104 may be configured tooperate within a business or other entity to perform a variety ofservices for the business or other entity. For example, client computers101-104 may be configured to operate as a web server, firewall, clientapplication, media player, mobile telephone, game console, desktopcomputer, or the like. However, client computers 101-104 are notconstrained to these services and may also be employed, for example, asfor end-user computing in other embodiments. It should be recognizedthat more or less client computers (as shown in FIG. 1) may be includedwithin a system such as described herein, and embodiments are thereforenot constrained by the number or type of client computers employed.

Computers that may operate as client computer 102 may include computersthat typically connect using a wired or wireless communications mediumsuch as personal computers, multiprocessor systems, microprocessor-basedor programmable electronic devices, network PCs, or the like. In someembodiments, client computers 101-104 may include virtually any portablecomputer capable of connecting to another computer and receivinginformation such as, laptop computer 102, mobile computer 104, tabletcomputers 103, or the like. However, portable computers are not solimited and may also include other portable computers such as cellulartelephones, display pagers, radio frequency (RF) devices, infrared (IR)devices, Personal Digital Assistants (PDAs), handheld computers,wearable computers, integrated devices combining one or more of thepreceding computers, or the like. As such, client computers 101-104typically range widely in terms of capabilities and features. Moreover,client computers 101-104 may access various computing applications,including a browser, or other web-based application.

A web-enabled client computer may include a browser application that isconfigured to send requests and receive responses over the web. Thebrowser application may be configured to receive and display graphics,text, multimedia, and the like, employing virtually any web-basedlanguage. In one embodiment, the browser application is enabled toemploy JavaScript, HyperText Markup Language (HTML), eXtensible MarkupLanguage (XML), JavaScript Object Notation (JSON), Cascading StyleSheets (CS S), or the like, or combination thereof, to display and senda message. In one embodiment, a user of the client computer may employthe browser application to perform various activities over a network(online). However, another application may also be used to performvarious online activities.

Client computers 101-104 also may include at least one other clientapplication that is configured to receive or send content betweenanother computer. The client application may include a capability tosend or receive content, or the like. The client application may furtherprovide information that identifies itself, including a type,capability, name, and the like. In one embodiment, client computers102-105 may uniquely identify themselves through any of a variety ofmechanisms, including an Internet Protocol (IP) address, a phone number,Mobile Identification Number (MIN), an electronic serial number (ESN), aclient certificate, or other device identifier. Such information may beprovided in one or more network packets, or the like, sent between otherclient computers, file system management server computer 116, or othercomputers.

Client computers 101-104 may further be configured to include a clientapplication that enables an end-user to log into an end-user accountthat may be managed by another computer, such as operations managementserver computer 116, or the like. Such an end-user account, in onenon-limiting example, may be configured to enable the end-user to manageone or more online activities, including in one non-limiting example,project management, software development, system administration,configuration management, search activities, social networkingactivities, browse various websites, communicate with other users, orthe like. Also, client computers may be arranged to enable users todisplay reports, or interactive user-interfaces.

Wireless network 110 is configured to couple client computers 102-104and its components with network 110. Wireless network 110 may includeany of a variety of wireless sub-networks that may further overlaystand-alone ad-hoc networks, and the like, to provide aninfrastructure-oriented connection for client computers 102-104. Suchsub-networks may include mesh networks, Wireless LAN (WLAN) networks,cellular networks, and the like. In one embodiment, the system mayinclude more than one wireless network.

Wireless network 110 may further include an autonomous system ofterminals, gateways, routers, and the like connected by wireless radiolinks, and the like. These connectors may be configured to move freelyand randomly and organize themselves arbitrarily, such that the topologyof wireless network 110 may change rapidly.

Wireless network 110 may further employ a plurality of accesstechnologies including 2nd (2G), 3rd (3G), 4th (4G) 5th (5G) generationradio access for cellular systems, WLAN, Wireless Router (WR) mesh, andthe like. Access technologies such as 2G, 3G, 4G, 5G, and future accessnetworks may enable wide area coverage for mobile computers, such asclient computers 102-104 with various degrees of mobility. In onenon-limiting example, wireless network 108 may enable a radio connectionthrough a radio network access such as Global System for Mobilecommunication (GSM), General Packet Radio Services (GPRS), Enhanced DataGSM Environment (EDGE), code division multiple access (CDMA), timedivision multiple access (TDMA), Wideband Code Division Multiple Access(WCDMA), High Speed Downlink Packet Access (HSDPA), Long Term Evolution(LTE), and the like. In essence, wireless network 110 may includevirtually any wireless communication mechanism by which information maytravel between client computers 102-104 and another computer, network, acloud-based network, a cloud instance, or the like.

Network 111 is configured to couple network computers with othercomputers, including, operations management server computer 116, clientcomputers 101, and client computers 102-104 through wireless network110, or the like. Network 111 is enabled to employ any form of computerreadable media for communicating information from one electronic deviceto another. Also, network 111 can include the Internet in addition tolocal area networks (LANs), wide area networks (WANs), directconnections, such as through a universal serial bus (USB) port, Ethernetport, other forms of computer-readable media, or any combinationthereof. On an interconnected set of LANs, including those based ondiffering architectures and protocols, a router acts as a link betweenLANs, enabling messages to be sent from one to another. In addition,communication links within LANs typically include twisted wire pair orcoaxial cable, while communication links between networks may utilizeanalog telephone lines, full or fractional dedicated digital linesincluding T1, T2, T3, and T4, or other carrier mechanisms including, forexample, E-carriers, Integrated Services Digital Networks (ISDNs),Digital Subscriber Lines (DSLs), wireless links including satellitelinks, or other communications links known to those skilled in the art.Moreover, communication links may further employ any of a variety ofdigital signaling technologies, including without limit, for example,DS-0, DS-1, DS-2, DS-3, DS-4, OC-3, OC-12, OC-48, or the like.Furthermore, remote computers and other related electronic devices couldbe remotely connected to either LANs or WANs via a modem and temporarytelephone link. In one embodiment, network 111 may be configured totransport information of an Internet Protocol (IP).

Additionally, communication media typically embodies computer readableinstructions, data structures, program modules, or other transportmechanisms and includes any information non-transitory delivery media ortransitory delivery media. By way of example, communication mediaincludes wired media such as twisted pair, coaxial cable, fiber optics,wave guides, and other wired media and wireless media such as acoustic,RF, infrared, and other wireless media.

Also, one embodiment of operations management server computer 116 isdescribed in more detail below in conjunction with FIG. 3. Although FIG.1 illustrates operations management server computer 116, or the like,each as a single computer, the innovations or embodiments are not solimited. For example, one or more functions of operations managementserver computer 116, or the like, may be distributed across one or moredistinct network computers. Moreover, in one or more embodiments,operations management server computer 116 may be implemented using aplurality of network computers. Further, in one or more of the variousembodiments, operations management server computer 116, or the like, maybe implemented using one or more cloud instances in one or more cloudnetworks. Accordingly, these innovations and embodiments are not to beconstrued as being limited to a single environment, and otherconfigurations, and other architectures are also envisaged.

Illustrative Client Computer

FIG. 2 shows one embodiment of client computer 200 that may include manymore or less components than those shown. Client computer 200 mayrepresent, for example, one or more embodiment of mobile computers orclient computers shown in FIG. 1.

Client computer 200 may include processor 202 in communication withmemory 204 via bus 228. Client computer 200 may also include powersupply 230, network interface 232, audio interface 256, display 250,keypad 252, illuminator 254, video interface 242, input/output interface238, haptic interface 264, global positioning systems (GPS) receiver258, open air gesture interface 260, temperature interface 262,camera(s) 240, projector 246, pointing device interface 266,processor-readable stationary storage device 234, and processor-readableremovable storage device 236. Client computer 200 may optionallycommunicate with a base station (not shown), or directly with anothercomputer. And in one embodiment, although not shown, a gyroscope may beemployed within client computer 200 to measure or maintain anorientation of client computer 200.

Power supply 230 may provide power to client computer 200. Arechargeable or non-rechargeable battery may be used to provide power.The power may also be provided by an external power source, such as anAC adapter or a powered docking cradle that supplements or recharges thebattery.

Network interface 232 includes circuitry for coupling client computer200 to one or more networks, and is constructed for use with one or morecommunication protocols and technologies including, but not limited to,protocols and technologies that implement any portion of the OSI modelfor mobile communication (GSM), CDMA, time division multiple access(TDMA), UDP, TCP/IP, SMS, MMS, GPRS, WAP, UWB, WiMax, SIP/RTP, GPRS,EDGE, WCDMA, LTE, UMTS, OFDM, CDMA2000, EV-DO, HSDPA, or any of avariety of other wireless communication protocols. Network interface 232is sometimes known as a transceiver, transceiving device, or networkinterface card (MC).

Audio interface 256 may be arranged to produce and receive audio signalssuch as the sound of a human voice. For example, audio interface 256 maybe coupled to a speaker and microphone (not shown) to enabletelecommunication with others or generate an audio acknowledgment forsome action. A microphone in audio interface 256 can also be used forinput to or control of client computer 200, e.g., using voicerecognition, detecting touch based on sound, and the like.

Display 250 may be a liquid crystal display (LCD), gas plasma,electronic ink, light emitting diode (LED), Organic LED (OLED) or anyother type of light reflective or light transmissive display that can beused with a computer. Display 250 may also include a touch interface 244arranged to receive input from an object such as a stylus or a digitfrom a human hand, and may use resistive, capacitive, surface acousticwave (SAW), infrared, radar, or other technologies to sense touch orgestures.

Projector 246 may be a remote handheld projector or an integratedprojector that is capable of projecting an image on a remote wall or anyother reflective object such as a remote screen.

Video interface 242 may be arranged to capture video images, such as astill photo, a video segment, an infrared video, or the like. Forexample, video interface 242 may be coupled to a digital video camera, aweb-camera, or the like. Video interface 242 may comprise a lens, animage sensor, and other electronics. Image sensors may include acomplementary metal-oxide-semiconductor (CMOS) integrated circuit,charge-coupled device (CCD), or any other integrated circuit for sensinglight.

Keypad 252 may comprise any input device arranged to receive input froma user. For example, keypad 252 may include a push button numeric dial,or a keyboard. Keypad 252 may also include command buttons that areassociated with selecting and sending images.

Illuminator 254 may provide a status indication or provide light.Illuminator 254 may remain active for specific periods of time or inresponse to event messages. For example, when illuminator 254 is active,it may back-light the buttons on keypad 252 and stay on while the clientcomputer is powered. Also, illuminator 254 may back-light these buttonsin various patterns when particular actions are performed, such asdialing another client computer. Illuminator 254 may also cause lightsources positioned within a transparent or translucent case of theclient computer to illuminate in response to actions.

Further, client computer 200 may also comprise hardware security module(HSM) 268 for providing additional tamper resistant safeguards forgenerating, storing or using security/cryptographic information such as,keys, digital certificates, passwords, passphrases, two-factorauthentication information, or the like. In some embodiments, hardwaresecurity module may be employed to support one or more standard publickey infrastructures (PKI), and may be employed to generate, manage, orstore key pairs, or the like. In some embodiments, HSM 268 may be astand-alone computer, in other cases, HSM 268 may be arranged as ahardware card that may be added to a client computer.

Client computer 200 may also comprise input/output interface 238 forcommunicating with external peripheral devices or other computers suchas other client computers and network computers. The peripheral devicesmay include an audio headset, virtual reality headsets, display screenglasses, remote speaker system, remote speaker and microphone system,and the like. Input/output interface 238 can utilize one or moretechnologies, such as Universal Serial Bus (USB), Infrared, WiFi, WiMax,Bluetooth™, and the like.

Input/output interface 238 may also include one or more sensors fordetermining geolocation information (e.g., GPS), monitoring electricalpower conditions (e.g., voltage sensors, current sensors, frequencysensors, and so on), monitoring weather (e.g., thermostats, barometers,anemometers, humidity detectors, precipitation scales, or the like), orthe like. Sensors may be one or more hardware sensors that collect ormeasure data that is external to client computer 200.

Haptic interface 264 may be arranged to provide tactile feedback to auser of the client computer. For example, the haptic interface 264 maybe employed to vibrate client computer 200 in a particular way whenanother user of a computer is calling. Temperature interface 262 may beused to provide a temperature measurement input or a temperaturechanging output to a user of client computer 200. Open air gestureinterface 260 may sense physical gestures of a user of client computer200, for example, by using single or stereo video cameras, radar, agyroscopic sensor inside a computer held or worn by the user, or thelike. Camera 240 may be used to track physical eye movements of a userof client computer 200.

GPS transceiver 258 can determine the physical coordinates of clientcomputer 200 on the surface of the Earth, which typically outputs alocation as latitude and longitude values. GPS transceiver 258 can alsoemploy other geo-positioning mechanisms, including, but not limited to,triangulation, assisted GPS (AGPS), Enhanced Observed Time Difference(E-OTD), Cell Identifier (CI), Service Area Identifier (SAI), EnhancedTiming Advance (ETA), Base Station Subsystem (BSS), or the like, tofurther determine the physical location of client computer 200 on thesurface of the Earth. It is understood that under different conditions,GPS transceiver 258 can determine a physical location for clientcomputer 200. In one or more embodiments, however, client computer 200may, through other components, provide other information that may beemployed to determine a physical location of the client computer,including for example, a Media Access Control (MAC) address, IP address,and the like.

In one or more of the various embodiments, applications, such as,operating system 206, other client apps 224, web browser 226, or thelike, may be arranged to employ geo-location information to select oneor more localization features, such as, time zones, languages,currencies, calendar formatting, or the like. Localization features maybe used in display objects, user-interfaces, reports, as well asinternal processes or databases. In one or more of the variousembodiments, geo-location information used for selecting localizationinformation may be provided by GPS 258. Also, in some embodiments,geolocation information may include information provided using one ormore geolocation protocols over the networks, such as, wireless network110 or network 111.

Human interface components can be peripheral devices that are physicallyseparate from client computer 200, allowing for remote input or outputto client computer 200. For example, information routed as describedhere through human interface components such as display 250 or keyboard252 can instead be routed through network interface 232 to appropriatehuman interface components located remotely. Examples of human interfaceperipheral components that may be remote include, but are not limitedto, audio devices, pointing devices, keypads, displays, cameras,projectors, and the like. These peripheral components may communicateover a Pico Network such as Bluetooth™, Zigbee™ and the like. Onenon-limiting example of a client computer with such peripheral humaninterface components is a wearable computer, which might include aremote pico projector along with one or more cameras that remotelycommunicate with a separately located client computer to sense a user'sgestures toward portions of an image projected by the pico projectoronto a reflected surface such as a wall or the user's hand.

A client computer may include web browser application 226 that isconfigured to receive and to send web pages, web-based messages,graphics, text, multimedia, and the like. The client computer's browserapplication may employ virtually any programming language, including awireless application protocol messages (WAP), and the like. In one ormore embodiments, the browser application is enabled to employ HandheldDevice Markup Language (HDML), Wireless Markup Language (WML),WMLScript, JavaScript, Standard Generalized Markup Language (SGML),HyperText Markup Language (HTML), eXtensible Markup Language (XML),HTMLS, and the like.

Memory 204 may include RAM, ROM, or other types of memory. Memory 204illustrates an example of computer-readable storage media (devices) forstorage of information such as computer-readable instructions, datastructures, program modules or other data. Memory 204 may store BIOS 208for controlling low-level operation of client computer 200. The memorymay also store operating system 206 for controlling the operation ofclient computer 200. It will be appreciated that this component mayinclude a general-purpose operating system such as a version of UNIX, orLINUX™, or a specialized client computer communication operating systemsuch as Windows Phone™, or the Symbian® operating system. The operatingsystem may include, or interface with a Java virtual machine module thatenables control of hardware components or operating system operationsvia Java application programs.

Memory 204 may further include one or more data storage 210, which canbe utilized by client computer 200 to store, among other things,applications 220 or other data. For example, data storage 210 may alsobe employed to store information that describes various capabilities ofclient computer 200. The information may then be provided to anotherdevice or computer based on any of a variety of methods, including beingsent as part of a header during a communication, sent upon request, orthe like. Data storage 210 may also be employed to store socialnetworking information including address books, buddy lists, aliases,user profile information, or the like. Data storage 210 may furtherinclude program code, data, algorithms, and the like, for use by aprocessor, such as processor 202 to execute and perform actions. In oneembodiment, at least some of data storage 210 might also be stored onanother component of client computer 200, including, but not limited to,non-transitory processor-readable removable storage device 236,processor-readable stationary storage device 234, or even external tothe client computer.

Applications 220 may include computer executable instructions which,when executed by client computer 200, transmit, receive, or otherwiseprocess instructions and data. Applications 220 may include, forexample, other client applications 224, web browser 226, or the like.Client computers may be arranged to exchange communications one or moreservers.

Other examples of application programs include calendars, searchprograms, email client applications, IM applications, SMS applications,Voice Over Internet Protocol (VOIP) applications, contact managers, taskmanagers, transcoders, database programs, word processing programs,security applications, spreadsheet programs, games, search programs,visualization applications, and so forth.

Additionally, in one or more embodiments (not shown in the figures),client computer 200 may include an embedded logic hardware deviceinstead of a CPU, such as, an Application Specific Integrated Circuit(ASIC), Field Programmable Gate Array (FPGA), Programmable Array Logic(PAL), or the like, or combination thereof. The embedded logic hardwaredevice may directly execute its embedded logic to perform actions. Also,in one or more embodiments (not shown in the figures), client computer200 may include one or more hardware micro-controllers instead of CPUs.In one or more embodiments, the one or more micro-controllers maydirectly execute their own embedded logic to perform actions and accessits own internal memory and its own external Input and Output Interfaces(e.g., hardware pins or wireless transceivers) to perform actions, suchas System On a Chip (SOC), or the like.

Illustrative Network Computer

FIG. 3 shows one embodiment of network computer 300 that may be includedin a system implementing one or more of the various embodiments. Networkcomputer 300 may include many more or less components than those shownin FIG. 3. However, the components shown are sufficient to disclose anillustrative embodiment for practicing these innovations. Networkcomputer 300 may represent, for example, one or more embodiments of aoperations management server such as operations management servercomputer 116, or the like, of FIG. 1.

Network computers, such as, network computer 300 may include a processor302 that may be in communication with a memory 304 via a bus 328. Insome embodiments, processor 302 may be comprised of one or more hardwareprocessors, or one or more processor cores. In some cases, one or moreof the one or more processors may be specialized processors designed toperform one or more specialized actions, such as, those describedherein. Network computer 300 also includes a power supply 330, networkinterface 332, audio interface 356, display 350, keyboard 352,input/output interface 338, processor-readable stationary storage device334, and processor-readable removable storage device 336. Power supply330 provides power to network computer 300.

Network interface 332 includes circuitry for coupling network computer300 to one or more networks, and is constructed for use with one or morecommunication protocols and technologies including, but not limited to,protocols and technologies that implement any portion of the OpenSystems Interconnection model (OSI model), global system for mobilecommunication (GSM), code division multiple access (CDMA), time divisionmultiple access (TDMA), user datagram protocol (UDP), transmissioncontrol protocol/Internet protocol (TCP/IP), Short Message Service(SMS), Multimedia Messaging Service (MMS), general packet radio service(GPRS), WAP, ultra-wide band (UWB), IEEE 802.16 WorldwideInteroperability for Microwave Access (WiMax), Session InitiationProtocol/Real-time Transport Protocol (SIP/RTP), or any of a variety ofother wired and wireless communication protocols. Network interface 332is sometimes known as a transceiver, transceiving device, or networkinterface card (NIC). Network computer 300 may optionally communicatewith a base station (not shown), or directly with another computer.

Audio interface 356 is arranged to produce and receive audio signalssuch as the sound of a human voice. For example, audio interface 356 maybe coupled to a speaker and microphone (not shown) to enabletelecommunication with others or generate an audio acknowledgment forsome action. A microphone in audio interface 356 can also be used forinput to or control of network computer 300, for example, using voicerecognition.

Display 350 may be a liquid crystal display (LCD), gas plasma,electronic ink, light emitting diode (LED), Organic LED (OLED) or anyother type of light reflective or light transmissive display that can beused with a computer. In some embodiments, display 350 may be a handheldprojector or pico projector capable of projecting an image on a wall orother object.

Network computer 300 may also comprise input/output interface 338 forcommunicating with external devices or computers not shown in FIG. 3.Input/output interface 338 can utilize one or more wired or wirelesscommunication technologies, such as USB™, Firewire™, WiFi, WiMax,Thunderbolt™, Infrared, Bluetooth™, Zigbee™, serial port, parallel port,and the like.

Also, input/output interface 338 may also include one or more sensorsfor determining geolocation information (e.g., GPS), monitoringelectrical power conditions (e.g., voltage sensors, current sensors,frequency sensors, and so on), monitoring weather (e.g., thermostats,barometers, anemometers, humidity detectors, precipitation scales, orthe like), or the like. Sensors may be one or more hardware sensors thatcollect or measure data that is external to network computer 300. Humaninterface components can be physically separate from network computer300, allowing for remote input or output to network computer 300. Forexample, information routed as described here through human interfacecomponents such as display 350 or keyboard 352 can instead be routedthrough the network interface 332 to appropriate human interfacecomponents located elsewhere on the network. Human interface componentsinclude any component that allows the computer to take input from, orsend output to, a human user of a computer. Accordingly, pointingdevices such as mice, styluses, track balls, or the like, maycommunicate through pointing device interface 358 to receive user input.

GPS transceiver 340 can determine the physical coordinates of networkcomputer 300 on the surface of the Earth, which typically outputs alocation as latitude and longitude values. GPS transceiver 340 can alsoemploy other geo-positioning mechanisms, including, but not limited to,triangulation, assisted GPS (AGPS), Enhanced Observed Time Difference(E-OTD), Cell Identifier (CI), Service Area Identifier (SAI), EnhancedTiming Advance (ETA), Base Station Subsystem (BSS), or the like, tofurther determine the physical location of network computer 300 on thesurface of the Earth. It is understood that under different conditions,GPS transceiver 340 can determine a physical location for networkcomputer 300. In one or more embodiments, however, network computer 300may, through other components, provide other information that may beemployed to determine a physical location of the client computer,including for example, a Media Access Control (MAC) address, IP address,and the like.

In one or more of the various embodiments, applications, such as,operating system 306, ingestion engine 322, clustering engine 324, webservices 329, or the like, may be arranged to employ geo-locationinformation to select one or more localization features, such as, timezones, languages, currencies, currency formatting, calendar formatting,or the like. Localization features may be used in user interfaces,dashboards, reports, as well as internal processes or databases. In oneor more of the various embodiments, geo-location information used forselecting localization information may be provided by GPS 340. Also, insome embodiments, geolocation information may include informationprovided using one or more geolocation protocols over the networks, suchas, wireless network 110 or network 111.

Memory 304 may include Random Access Memory (RAM), Read-Only Memory(ROM), or other types of memory. Memory 304 illustrates an example ofcomputer-readable storage media (devices) for storage of informationsuch as computer-readable instructions, data structures, program modulesor other data. Memory 304 stores a basic input/output system (BIOS) 308for controlling low-level operation of network computer 300. The memoryalso stores an operating system 306 for controlling the operation ofnetwork computer 300. It will be appreciated that this component mayinclude a general-purpose operating system such as a version of UNIX, orLINUX, or a specialized operating system such as Microsoft Corporation'sWindows® operating system, or Apple Corporation's OSX® operating system.The operating system may include, or interface with one or more virtualmachine modules, such as, a Java virtual machine module that enablescontrol of hardware components or operating system operations via Javaapplication programs. Likewise, other runtime environments may beincluded.

Memory 304 may further include one or more data storage 310, which canbe utilized by network computer 300 to store, among other things,applications 320 or other data. For example, data storage 310 may alsobe employed to store information that describes various capabilities ofnetwork computer 300. The information may then be provided to anotherdevice or computer based on any of a variety of methods, including beingsent as part of a header during a communication, sent upon request, orthe like. Data storage 310 may also be employed to store socialnetworking information including address books, friend lists, aliases,user profile information, or the like. Data storage 310 may furtherinclude program code, data, algorithms, and the like, for use by aprocessor, such as processor 302 to execute and perform actions such asthose actions described below. In one embodiment, at least some of datastorage 310 might also be stored on another component of networkcomputer 300, including, but not limited to, non-transitory media insideprocessor-readable removable storage device 336, processor-readablestationary storage device 334, or any other computer-readable storagedevice within network computer 300, or even external to network computer300. Data storage 310 may include, for example, learner objects 312,event data 314, or the like.

Applications 320 may include computer executable instructions which,when executed by network computer 300, transmit, receive, or otherwiseprocess messages (e.g., SMS, Multimedia Messaging Service (MMS), InstantMessage (IM), email, or other messages), audio, video, and enabletelecommunication with another user of another mobile computer. Otherexamples of application programs include calendars, search programs,email client applications, IM applications, SMS applications, Voice OverInternet Protocol (VOIP) applications, contact managers, task managers,transcoders, database programs, word processing programs, securityapplications, spreadsheet programs, games, search programs, and soforth. Applications 320 may include ingestion engine 322, clusteringengine 324, web services 329, or the like, that may be arranged toperform actions for embodiments described below. In one or more of thevarious embodiments, one or more of the applications may be implementedas modules or components of another application or engine. Further, inone or more of the various embodiments, applications may be implementedas operating system extensions, modules, plugins, or the like.

Furthermore, in one or more of the various embodiments, ingestion engine322, clustering engine 324, web services 329, or the like, may beoperative in a cloud-based computing environment. In one or more of thevarious embodiments, these applications, and others, that comprise themanagement platform may be executing within virtual machines or virtualservers that may be managed in a cloud-based based computingenvironment. In one or more of the various embodiments, in this contextthe applications may flow from one physical network computer within thecloud-based environment to another depending on performance and scalingconsiderations automatically managed by the cloud computing environment.Likewise, in one or more of the various embodiments, virtual machines orvirtual servers dedicated to ingestion engine 322, clustering engine324, web services 329, or the like, may be provisioned andde-commissioned automatically.

Also, in one or more of the various embodiments, ingestion engine 322,clustering engine 324, web services 329, or the like, may be located invirtual servers running in a cloud-based computing environment ratherthan being tied to one or more specific physical network computers.

Further, network computer 300 may also comprise hardware security module(HSM) 360 for providing additional tamper resistant safeguards forgenerating, storing or using security/cryptographic information such as,keys, digital certificates, passwords, passphrases, two-factorauthentication information, or the like. In some embodiments, hardwaresecurity module may be employed to support one or more standard publickey infrastructures (PKI), and may be employed to generate, manage, orstore key pairs, or the like. In some embodiments, HSM 360 may be astand-alone network computer, in other cases, HSM 360 may be arranged asa hardware card that may be installed in a network computer.

Additionally, in one or more embodiments (not shown in the figures),network computer 300 may include an embedded logic hardware deviceinstead of a CPU, such as, an Application Specific Integrated Circuit(ASIC), Field Programmable Gate Array (FPGA), Programmable Array Logic(PAL), or the like, or combination thereof. The embedded logic hardwaredevice may directly execute its embedded logic to perform actions. Also,in one or more embodiments (not shown in the figures), the networkcomputer may include one or more hardware microcontrollers instead of aCPU. In one or more embodiments, the one or more microcontrollers maydirectly execute their own embedded logic to perform actions and accesstheir own internal memory and their own external Input and OutputInterfaces (e.g., hardware pins or wireless transceivers) to performactions, such as System On a Chip (SOC), or the like.

Illustrative Logical System Architecture

FIG. 4 illustrates a logical architecture of system 400 for inlinecategorizing of events in accordance with one or more of the variousembodiments. In one or more of the various embodiments, a system forinline categorizing of events may comprise various components. In thisexample, system 400 includes, ingestion engine 402, clustering engine404, database 406, service A 408, service B 410, service ZZ 412, servicedata 414, service data 416, service data 418, response manager 420,response manager 422, response manager 424, user/manager feedback 426,learner objects 428, event 430, event 432, event 434, or the like.

In one or more of the various embodiments, an ingestion engine such asingestion engine 402 may be arranged to receive or obtain one or moredifferent types of events provided by various sources, here representedby event 430, event 432, and event 434. In one or more of the variousembodiments, events may be variously formatted messages that reflect theoccurrence of events or incidents that have occurred in anorganization's computing system. Such events may include alertsregarding system errors, warnings, failure reports, customer servicerequests, status messages, or the like. Events may be collected by oneor more external services and provided to system 400. Events, asdescribed above may be comprised of SMS messages, HTTP requests/posts,API calls, log file entries, trouble tickets, emails, or the like. In atleast one of the various embodiments, events may include associatedinformation, such as, source, time stamps, status indicators, or thelike, that may be tracked. Also, in some embodiments, events, may alsobe associated with one or more service teams that may be responsible forresolving the issues related to the events.

Accordingly, ingestion engine 402 may be arranged to receive the variousevents and perform various actions, including, filtering, reformatting,information extraction, data normalizing, or the like, or combinationthereof, to enable the events to be stored and processed. In one or moreof the various embodiments, information associated with events or theevents themselves may be stored in database 406.

In one or more of the various embodiments, events may be provided by oneor more organizations. In some embodiments, there may be severalorganization (e.g., 100's, 1000's, or the like) that provide events tothe system. Events from different organizations may be segregated fromeach other so that an organization may only interact with events thatare owned by it. However, system 400 may be arranged to have visibilityto all of the events enabling community wide analysis to be performed.

In one or more of the various embodiments, ingestion engine 402 may bearranged to normalize incoming events into a unified common eventformat. Accordingly, in some embodiments, ingestion engine 402 may bearranged to employ configuration information, including, rules,templates, maps, dictionaries, or the like, or combination thereof, tonormalize the fields and values of incoming events to the common eventformat.

In one or more of the various embodiments, clustering engine 404, may bearranged to execute one or more clustering processes to group events. Asdescribed in more detail below, clustering engine 404 may be arranged togroup events into event groups based on one or more characteristics ofthe events.

In one or more of the various embodiments, clustering engine 404 may bearranged to group events to enable them to be provided to one or moreoperations management services, such as, service A 408, service B 410,service ZZ 412, or the like. In some embodiments, services may beconfigured by users or organizations to collect events or manageincidents for one or more applications, services, or areas of operation,or the like, of an organization. In this example, for some embodiments,service data stores, such as, service data 414, service data 416,service data 418, or the like, represent events or event informationthat may be stored or collected for a given service. In someembodiments, the various data stores may be stored on a single database.Likewise, in some embodiments, the data stores may be distributed orseparated from each other.

Accordingly, in one or more of the various embodiments, routing eventsto the applicable service enables one or more responsible users toresolve or otherwise respond to events. In some embodiments, responsemanager 420, response manager 422, response manager 424, or the like,represent one or more applications, such as, incident managementapplications, or the like, that may be monitored or managed by one ormore users.

Further, in one or more of the various embodiments, system 400 may bearranged to include one or more applications, user interfaces, or thelike, that enable users, administrators, response managers, or the like,to provide feedback associated with the grouping of events. Accordingly,in one or more of the various embodiments, users may provide feedbackinformation that indicates if an event grouping or non-grouping may beincorrect. For example, in some embodiments, operations managementsystems, such as, system 400 may be arranged to enable users to reviewevent groupings and provide input that indicates an event associatedwith a group may be incorrect. Likewise, in some embodiments, system 400may be arranged to enable users to provide input that indicates an eventthat was not associated with a group should have been associated withthe group.

In one or more of the various embodiments, feedback from users regardingthe quality of event grouping may be captured to generate one or morelearner objects, such as learner objects 428. In one or more of thevarious embodiments, learner objects may be data structures that may beassociated with users, services, accounts, response managers, or thelike. For example, in some embodiments, each user may have its ownaccount or each service may be associated with one or more accounts. Inother embodiments, an organization may have one learner object for theentire organization. Accordingly in one or more of the variousembodiments, clustering engines may be arranged to employ rules,instructions, or the like, provided via configuration information todetermine how learner objects may be associated with users, services,accounts, organizations, or the like.

In one or more of the various embodiments, as events arrive to system400, they may be processed to identify or determine one or more textstring components of a given event. In some embodiments, the entireevent may be a text string, such as, a log record, email, text message,or the like. Also, in some embodiments, event text may be embedded in aother data structures, encrypted, compressed, encoded, or the like.Accordingly, in one or more of the various embodiments, ingestionengines may be arranged to perform one or more actions to determine thetext strings included in an event. In one or more of the variousembodiments, ingestion engines may be arranged to employ parsers,grammars, rules, filters, templates, or the like, provided viaconfiguration information to determine or extract text information(e.g., message information) from incoming events.

Accordingly, in one or more of the various embodiments, the messageinformation may be provided to a clustering engine, such as, clusteringengine 404 for additional processing.

In one or more of the various embodiments, clustering engine 404 may bearranged to generate a message vector based on the message information.Accordingly, in some embodiments, the message vector may be comparedwith one or more group vectors to determine a similarity score thatrepresents how close the message vector matches each group vector. Insome embodiments, if one or more of the similarity scores exceed adefined threshold value, the event associated with the message vectormay be associated with the one or more event groups that may beassociated with one or more group vectors determined to be similar tothe message vector.

In one or more of the various embodiments, if a message vector may bedetermined to be similar to a group vector, the message vector may beadded (component-wise) to that group vector. Accordingly, in someembodiments, as similarities are determined, the group vector may betuned or refined based on incorporating the component values of similarmessage vectors in the group vector.

Further, in one or more of the various embodiments, users may be enabledto provide feedback regarding the grouping of events. Accordingly, insome embodiments, users may employ one or more user interfaces to gradeor otherwise indicate their agreement or disagreement with the groupingof individual events.

Accordingly, in one or more of the various embodiments, clusteringengines may be arranged to associate learner objects with users oraccounts to capture the user feedback for incorporation into groupingevents. In some embodiments, learner objects may include informationthat may be employed to generate a score that may indicate if thelearner object agrees with the grouping decision made by the clusteringengine. Thus, in some embodiments, clustering engines may be arranged toevaluate message vectors using a learner object associated with the useror organization to generate an agreement score that may be employed tooverride a grouping decision.

In one or more of the various embodiments, clustering engines may bearranged to store a record of similarity overrides so they may bereviewed by users or organizations as needed.

FIG. 5 illustrates a logical flow of a portion of the operations ofclustering engine 500 for inline categorizing of events in accordancewith one or more of the various embodiments. As described above, in someembodiments, clustering engines may be provided message informationassociated with events.

In this example, for some embodiments, message information may beprovided to a clustering engine. In some embodiments, the event textinformation may be provided as is or after initial processing has beenperformed by an ingestion engine. In one or more of the variousembodiments, the message information may be in one or more datastructures or data packets provided via one or more interfaces,databases, streams, APIs, or the like. In one or more of the variousembodiments, the particular delivery mechanism may vary depending onlocal circumstances, local requirements, event providers, or the like.Accordingly, in some embodiments, clustering engines may be arranged toemploy instructions, rules, or the like, provided via configurationinformation to manage message information delivery.

At step 502, in some embodiments, the text string may be considered torepresent text associated with an event generated for an organizationregarding one or more network services.

In one or more of the various embodiments, clustering engines may bearranged to remove portions of the text associated with individual orunique characteristics that may be considered non-semantic because theremoved portions may not contribute to the meaning of the message.Rather, in some embodiments, the removed information may be associatedunique instances of messages that may otherwise have the same meaning.For example, in some embodiments, timestamps, GUIDs, IP addresses, userIDs, sequence numbers, or the like, may be determined to be removablenon-semantic information.

At step 504, in some embodiments, for some embodiments, the clusteringengine has removed the non-semantic information from the messageinformation. In some embodiments, clustering engines may be arranged toemploy one or more regular expressions, templates, filters, grammars,parsers, or the like, that may be provided or determined viaconfiguration information to remove undesirable non-semantic informationfrom text information.

At step 506, in some embodiments, for some embodiments, the clusteringengine has generated a list of the individual words included in the textinformation. In some embodiments, clustering engines may be arranged todetermine individual words from the message information. In someembodiments, clustering engines may be arranged to employ one or moreconventional or custom mechanisms to automatically generate the list ofindividual message words. Accordingly, in some embodiments, clusteringengines may be arranged to employ one or more regular expressions,grammars, parsers, filters, maps, or the like, to generate the list ofmessage words.

At step 508, in some embodiments, for some embodiments, the clusteringengine may be arranged to generate a list of 2-grams from the messagestring. In some embodiments, clustering engines may perform actionssimilar as described above, except each item in the 2-gram list may becomprised of two adjacent words paired together. As shown in FIG. 5,words may appear more than once in the 2-gram list, for example, thefirst item at step 508 is “Alert foreign” and second item is “foreignsystem” where the word “foreign” may be included in a 2-gram with itsleft adjacent neighbor and in another 2-gram with its right adjacentneighbor.

At step 510, in some embodiments, for some embodiments, the clusteringengine may be arranged to apply a hash function to each word in the listgenerated at step 506 and to each 2-gram generated at step 508. In someembodiments, clustering engines may be arranged to employ a hashfunction that generates key values for a defined key space. For example,in some embodiments, a clustering engine may be arranged to employ ahash function that given a string of any length produces a hash key thathas values from 0-2{circumflex over ( )}20 (e.g., 0-1,048,577). In someembodiments, the particular hash function or key size may be selectedbased on various factors, including, the particular application, localcircumstances, local requirements, predominant languages (locale),CPU/GPU characteristics, or the like. Accordingly, in one or more of thevarious embodiments, the clustering engines may be arranged to employrules, instructions, parameter values, or the like, provided viaconfiguration information to determine the hash function or key sizeemployed at step 510.

In one or more of the various embodiments, the clustering engine may bearranged to provide a message vector with the same number of componentsas the size of the key space of the hash function.

In one or more of the various embodiments, the clustering engine may bearranged to employ the key value associated with each message word ormessage 2-gram to determine an component in the message vector torepresent the word or 2-gram. Accordingly, in some embodiments, thevector component corresponding to each hash key may be set to a value ofone (1). For example, for some embodiments, if a hash function generatesa key value of 5670 for a message word, component 5670 in the messagevector may be set to one.

In one or more of the various embodiments, clustering engines may bearranged to represent the message vector using a sparse vector datastructure that includes a component for each hash key value and omitscomponents assumed to have a value of zero (0). For example, the messagevector shown at step 510 includes entries, such as, 4596:1, 5675:1, andso on. In this example, entry 4596:1 may represent that component 4596of the message vector is set to one (1). Likewise, in this example,entry 5675:1 may represent that component 5675:1 in the message vectoris set to one (1). In some embodiments, other data structures or memoryarrangements may be employed to represent message vectors. However, inthe interest of brevity and clarity, additional examples have beenomitted. And, one of ordinary skill in the art will appreciate that theprovided examples are at least sufficient to disclose the innovationsdescribed herein.

Accordingly, in one or more of the various embodiments, clusteringengines may be arranged to generate a message vector, as described basedon each event provided by the ingestion engine. In some embodiments,message vectors may be generated in real-time as events are received.Also, in one or more of the various embodiments, message vectors may begenerated off-line from one or more event archives, log files, or thelike.

FIG. 6 illustrates a logical flow of a portion of the operations ofclustering engine 600 for inline categorizing of events in accordancewith one or more of the various embodiments. As described above, in someembodiments, clustering engines may be arranged to generate messagevectors from events.

In one or more of the various embodiments, clustering engines may bearranged to determine the similarity between two message vectors basedthe cosine similarity between the two vectors. In some embodiments,generating the cosine similarity provides a scalar value from 0-1 thatrepresents the cosine of the angle between the two vectors. Accordingly,the cosine similarity for two vectors may be employed as a similarityscore to measure the similarity between two vectors. For example, if twovectors have the same orientation the cosine similarity value will 1.0and if they are oriented 90 degrees to each other, the cosine similaritywill be 0.0. Cosine similarity may be generated by performing a vectordot product of each vector and dividing that result by product of themagnitudes the two vectors.

Accordingly, at step 602, in some embodiments, a clustering engine maybe arranged to determine the similarity of message vector 602A andmessage vector 602B by generating a similarity score based on the cosinesimilarity of message vector 602A and message vector 602B.

At step 604, in some embodiments, a similarity score of 0.7 has beengenerated. Accordingly, in some embodiments, if this value exceeds asimilarity threshold, the two message vectors may be considered similar.

In one or more of the various embodiments, message vectors determined tobe similar may be considered a group (here a group of two). Accordingly,in one or more of the various embodiments, if two message vectors may bedetermined to be similar, clustering engines may be arranged to generatea group vector by adding the two message vectors together(component-wise).

In one or more of the various embodiments, clustering engines may bearranged to determine similarity scores for incoming message vectors andgroup vectors. Accordingly, in one or more of the various embodiments,message vectors determined to be sufficiently similar to one or moregroup vectors may be added to the determined one or more group vectors.Further, in one or more of the various embodiments, the eventsassociated with the message vectors may be associated with the eventgroups that may be associated with the group vectors.

Also, in one or more of the various embodiments, clustering engines maybe arranged to generate one or more reports that show event groupsassociated with a service. In some cases, in some embodiments, thereports may include unlabeled groups that were otherwise unknown.Accordingly, in some embodiments, users may be enabled name or labelthese newly discovered groups.

Further, in some embodiments, clustering engines may be arranged toevaluate message vectors or group vectors associated with differentservices (e.g., Service A 408 or Service B 410) for similarity.Accordingly, in one or more of the various embodiments, clusteringengines may enable efficient cross service event grouping that otherwisemay be unexpected or difficult to discover. For example, similarityscores generated for message vectors associated with one service andgroup vectors for another service may be evaluated to determine crossservice groupings or similarity.

Note, in one or more of the various embodiments, as new previouslyunseen events are provided, new groups may be automatically generated assimilar message vectors associated with the new events may bedetermined. Accordingly, in one or more of the various embodiments, thisprovides an advantageous improvement because the clustering engines maylearn new groups on-the-fly without requiring off-line training commonlyrequired for conventional machine learning classifiers.

FIGS. 7A and 7B illustrate the logical flows of a portion of theoperations of clustering engine 700 for inline categorizing of events inaccordance with one or more of the various embodiments. As describedabove, in some embodiments, clustering engines may be arranged togenerate learner object data structures that may be employed tointroduce user feedback into event grouping. In one or more of thevarious embodiments, clustering engines may be arranged to enable usersto associate one or more selected events with event groups that hadsimilarity scores that fell below the similarity threshold. Likewise, insome embodiments, clustering engines may be arranged to enable users todisassociate one or more selected events from groups that had similarityscores that were above the similarity threshold.

Accordingly, in one or more of the various embodiments, clusteringengines may be arranged to generate learner objects that may beassociated with users, accounts, organizations, or the like.Accordingly, in one or more of the various embodiments, a learner objectmay be adapted to particular users, organizations, or accounts.

FIG. 7A illustrates the logical flow for generating or adapting learnerobjects in accordance with one or more of the various embodiments.

At step 702, in some embodiments, a user may be disassociating an eventcorresponding to message vector 702A from a group corresponding to groupvector 702B.

In one or more of the various embodiments, the first time a learnerobject may be needed may be if a user selects an event to associated ordisassociated with a event group. For example, if a user indicates thatan event should be disassociated from a group it was previouslyassociated with, the clustering engine may be arranged to generate alearner object for that user.

In one or more of the various embodiments, if a user employs auser-interface to select an event they want to disassociate from anevent group, a clustering engine may be arranged to determine thecorresponding message vector and group vector that may be stored in aservice data store or other database. Similarly, in some embodiments, ifa user may select an event to associate with an event group, theclustering engine may determine the message vector and group vectorbased on the selected event and event group.

In this example, message vector 702A may be considered the messagevector for the selected event and group vector 702B may be consideredthe group vector for the event group of interest.

At step 704, in some embodiments, the clustering engine has generatedlearner object 704A based on message vector 702A and group vector 702B.

Accordingly, in one or more of the various embodiments, the clusteringengine may be arranged to produce a matrix based on the outer product ofmessage vector 702A and group vector 702B (e.g., message vector*groupvector). In some embodiments, if there is no previously created learnerobject, the clustering engine may provide a zero valued matrix as aninitial start value. Otherwise, in some embodiments, if the learnerobject was created previously, the clustering engine may provide amatrix from the existing learner object.

In one or more of the various embodiments, if the user may beassociating an event with an event group, the outer product of messagevector 702A and group vector 702B may be added to the learner objectmatrix (component-wise). Alternatively, if the user may bedisassociating an event from an event group, the outer product ofmessage vector 702A and group vector 702B may be subtracted from thelearner object matrix (component-wise). Accordingly, in someembodiments, as a user associates or disassociates events and eventgroups, their associated learner object may be updated. If it may be thefirst time a user associates or disassociates events and event groups,the learner object may be created and initialized with the event andevent group selected by the user.

In one or more of the various embodiments, learner objects may include amatrix that may be represented using a sparse matrix data structure,such that, zero valued components may be omitted. In some embodiments,other data structures or memory arrangements may be employed torepresent learner object matrices. However, in the interest of brevityand clarity, additional examples have been omitted. And, one of ordinaryskill in the art will appreciate that the provided examples are at leastsufficient to disclose the innovations described herein.

Accordingly, in one or more of the various embodiments, clusteringengines may be arranged to employ databases, service data stores, or thelike, to store learner objects and associate them with users, accounts,organizations, or the like.

FIG. 7B illustrates the logical flow for employing learner objects inaccordance with one or more of the various embodiments.

In one or more of the various embodiments, clustering engines may bearranged to employ learner objects to identify if users, or the like,have expressed an intent to override similarity grouping. Accordingly,in one or more of the various embodiments, as events may be provided toa clustering engine, learner objects, if available, may be employed togenerate an agreement score that indicates whether the learner objectagrees with a similarity determination.

In one or more of the various embodiments, agreement scores may begenerated for each message vector and group vector separately from thesimilarity scoring.

At step 706, in some embodiments, a message vector, such as, messagevector 706A may be provided for an incoming event. Accordingly, in someembodiments, the clustering engine may be arranged to select a groupvector, such as, group vector 702B that corresponds to the event groupbeing considered. Note, in one or more of the various embodiments, thegroup vector may be the group vector for the same group that was used togenerate or adapt the learner object. Further, in some embodiments, theclustering engine may determine a learner object for the user, such as,learner object 704A. In some embodiments, if the message vector, groupvector, and learner object have been determined, the clustering enginemay be arranged to generate the agreement score for the incoming messagevector.

At step 708, in some embodiments, the clustering engine may be arrangedto generate the agreement score. In one or more of the variousembodiments, clustering engines may be arranged to generate agreementscores for a message vector v and a group vector g by generating theouter product vg. This result may be employed as a mask for the learnerobject matrix L by generating the Kronecker product between the two andretaining those entries of L for which there is a nonzero entry in vg(the outer product the message vector and the group vector).Accordingly, the sum of these non-zero entries provides a scalar valuethat may be considered to be the agreement score.

In one or more of the various embodiments, if a learner object producesa large positive agreement score for an event and an event group, theclustering engine may be arranged to disregard a low similarity scoreand associate the event with the event group. Likewise, in one or moreof the various embodiments, if a learner object produces a largenegative agreement score for an event and an event group, the clusteringengine may be arranged to a disregard a high similarity score andrefrain from associating the event with the event group.

Generalized Operations

FIGS. 8-10 represent generalized operations for inline categorizing ofevents in accordance with one or more of the various embodiments. In oneor more of the various embodiments, processes 800, 900, and 1000described in conjunction with FIGS. 8-10 may be implemented by orexecuted by one or more processors on a single network computer, such asnetwork computer 300 of FIG. 3. In other embodiments, these processes,or portions thereof, may be implemented by or executed on a plurality ofnetwork computers, such as network computer 300 of FIG. 3. In yet otherembodiments, these processes, or portions thereof, may be implemented byor executed on one or more virtualized computers, such as, those in acloud-based environment. However, embodiments are not so limited andvarious combinations of network computers, client computers, or the likemay be utilized. Further, in one or more of the various embodiments, theprocesses described in conjunction with FIGS. 8-10 may perform actionsfor inline categorizing of events in accordance with one or more of thevarious embodiments or architectures such as those described inconjunction with FIGS. 4-7. Further, in one or more of the variousembodiments, some or all of the actions performed by processes 800, 900,and 1000 may be executed in part by ingestion engine 322, clusteringengine 324, or the like.

FIG. 8 illustrates an overview flowchart for process 800 for inlinecategorizing of events in accordance with one or more of the variousembodiments. After a start block, at block 802, in one or more of thevarious embodiments, one or more events with message information may beprovided to a clustering engine. At block 804, in one or more of thevarious embodiments, the clustering engine may be arranged to generatemessage vectors for the one or more events based on the messageinformation associated with each event. At block 806, in one or more ofthe various embodiments, the clustering engine may be arranged toprovide one or more group vectors. At block 808, in one or more of thevarious embodiments, the clustering engine may be arranged to generatesimilarity scores based on the message vector and the one or more groupvectors. At block 810, in one or more of the various embodiments, theclustering engine may be arranged to generate learner object agreementscores for the message vector and the group vectors. At block 812, inone or more of the various embodiments, the clustering engine may bearranged to associate the one or more events with one or more eventgroups based on the similarity scores and the agreement scores. Next, inone or more of the various embodiments, control may be returned to acalling process.

FIG. 9 illustrates a flowchart for process 900 for inline categorizingof events in accordance with one or more of the various embodiments.After a start block, at block 902, in one or more of the variousembodiments, message information may be provided to a clustering engine.As described above, ingestion engines may be provided events from avariety of event sources. The ingestion engines may be arranged toperform any necessary pre-processing of events to prepare or extractmessage information associated with the incoming events. Accordingly, inone or more of the various embodiments, message information provided toclustering engines may include the message text, as well, meta-data thatmay be employed to associate the message text with the original event.

At block 904, in one or more of the various embodiments, the clusteringengine may be arranged to remove the non-semantic information from themessage information. As described above, non-semantic information mayinclude IP addresses, GUIDs, MAC addresses, timestamps, serial numbers,sequence numbers, or the like, that may included in the message text.However, this information may introduce unnecessary or undesirableentropy that does not convey or relate to the meaning of the event forgrouping purposes. For example, in one or more of the variousembodiments, if an event's message text include a timestamp, eachoccurrence same type of event may appear less similar than expectedbecause each instance would have a different timestamp value.

Note, in some embodiments, the non-semantic information may remainassociated or included in the event or included as meta-data.Accordingly, in some embodiments, it may be available to other servicesor applications for sorting, filtering, or the like.

At block 906, in one or more of the various embodiments, the clusteringengine may be arranged to hash each message word and each message 2-gramincluded in the message information and store the generated hash keyvalues in a message vector. As described above, clustering engines maybe arranged to generate a list of individual words that may be includedin the message text. Likewise, in one or more of the variousembodiments, clustering engines may be arranged to generate another listthat includes 2-grams comprised of adjacent words in the message text.As shown in FIG. 5, the second word in the 2-gram may be used as thefirst word in the next 2-gram, and so on.

Accordingly, in some embodiments, clustering engines may be arranged toemploy a hash function to generate hash key values for each word in thesingle word list and for each 2-gram in the 2-gram list.

In one or more of the various embodiments, the hash key values may bestored in a sparse vector data structure, such that each hash key valueis treated as an component position in the vector and the value at thatposition in the vector may be set to one. The remaining components inthe vector may be considered to have a value of zero (0). In one or moreof the various embodiments, vectors for incoming events may beconsidered message vectors.

At block 908, in one or more of the various embodiments, clusteringengines may be arranged to generate similarity scores for messagevectors and group vectors. As described above, in some embodiments,clustering engines may be arranged to generate similarity scores foreach event and each event group. In some embodiments, clustering enginesmay be arranged to generate similarity scores based on the cosinesimilarity values for each message vector and group vector.

In some cases, in some embodiments, clustering engines may be arrangedto generate similarity scores between message vectors rather than beinglimited to generating similarity scores for message vectors and groupvectors. In some embodiments, clustering engines may be arranged togenerate similarity scores for message vectors to determine if there maybe new groups of message vectors. Also, in some embodiments, if aclustering engine is being initialized or put in use for the first time,there may not be any group vectors because the groups have yet to bedetermined. Accordingly, in some embodiments, similarity between messagevectors may be evaluated to discover new groups that may result in newgroup vectors.

At block 910, in one or more of the various embodiments, clusteringengines may be arranged to employ learner objects to generate agreementscores for message vectors and group vectors. In one or more of thevarious embodiments, learner objects may be associated with users,accounts, organizations, services, or the like. Accordingly, clusteringengines may be arranged to retrieve learner objects that may beassociated with pending operations. In some embodiments, this mayinclude a learner object associated with an administrative accountassociated with the event operations management server, or the like,including a user representing one or more services or processes ratherthan a user representing a person.

In some embodiments, in some cases, a relevant learner object may not beavailable. For example, if a user has not provided explicit or implicitfeedback associated with prior event grouping, a learner object may beunavailable for that user.

However, in some embodiments, if a relevant learner object may beavailable, it may be provided. Accordingly, in some embodiments,clustering engines may be arranged to generate agreement scores for eachmessage vector and group vector.

At decision block 912, in one or more of the various embodiments, if thesimilarity score exceeds a threshold value, control may flow to decisionblock 914; otherwise, control may flow to decision block 916. In someembodiments, clustering engines may be arranged to determine similarityscore threshold values based on configuration information to account forlocal circumstances or local requirements.

At decision block 914, in one or more of the various embodiments, if therelevant learner object may be in agreement, control may flow to block918; otherwise, control may be returned to calling process.

As described above, in some embodiments, a learner object's strength ofagreement or disagreement with a similarity score may be based on themagnitude of the agreement score. In some embodiments, if an agreementscore value may be positive, the learner object is indicating that amessage vector and group vector should be considered similar. And, insome embodiments, if an agreement score value may be a negative value,the learner object may be indicating it does not consider a messagevector and a group vector as being similar. In one or more of thevarious embodiments, clustering engines may be arranged to employ rules,threshold values, value ranges, or the like, provided via configurationinformation to determine if a learner object agreement score may bestrong or weak.

Accordingly, in some embodiments, if the similarity score exceeds thegrouping threshold value and the learner object agreement score is weak(positive or negative with a magnitude below a threshold value), controlmay flow to block 918. Similarly, if the learner object agreement scoreis strongly positive, control may flow to block 918. However, in someembodiments, if the learner object agreement score may be stronglynegative, control may be returned to calling process without associatingthe event with the group.

At decision block 916, in one or more of the various embodiments, if therelevant learner object may be in agreement, control may be returned acalling process; otherwise, control may flow to block 918.

In some embodiments, if the similarity score may be below the groupingthreshold value and the learner object agreement score is weak orstrongly negative, control may be returned to a calling process withoutassociating the event with the event group.

Alternatively, in some embodiments, if the learner object agreementscore may be strongly positive, control may flow to block 918, becausethe learner object agreement score may override the low similarityscore.

At block 918, in one or more of the various embodiments, the clusteringengine may be arranged to associate an event associated with the messageinformation to one or more event groups and update the one or more groupvectors based on the message vector. As described above, message vectorsdetermined to be similar group vectors may be added to the similar groupvectors.

Next, in one or more of the various embodiments, control may be returnedto a calling process.

FIG. 10 illustrates a flowchart for process 1000 for inline categorizingof events in accordance with one or more of the various embodiments.After a start block, at block 1002, in one or more of the variousembodiments, a clustering engine may be arranged to enable users toprovide feedback associated with event groupings. As described above,clustering engines may be arranged to provide one or more userinterfaces that enable users to review event groupings. In someembodiments, users may be enabled to provide feedback regarding thequality of event grouping. In some embodiments, users may be enabled todirectly move one or more events associated with one event group toanother group. In some embodiments, users may be enabled to disassociatean event from an event group while not associating it with a differentevent group.

In one or more of the various embodiments, clustering engines may bearranged to passively monitor whether users move events from one eventgroup to another. Accordingly, in some embodiments, such movement ofevents may be automatically considered user feedback.

At block 1004, in one or more of the various embodiments, the clusteringengine may be arranged to update a learner object for the user based onthe provided feedback. As described above, in some embodiments,clustering engines may be arranged to perform various actions togenerate new learner objects for users or update or adapt existinglearner objects based on the user feedback.

Accordingly, in some embodiments, if the user does not have anassociated learner object, the clustering engine may be arranged togenerate a new learner object for the user. Alternatively, in someembodiments, if there may already be a learner object for the user, theclustering engines may be arranged to modify the learner object to adaptto the user feedback.

Further, in one or more of the various embodiments, the clusteringengine may be arranged to determine the message vectors and the groupvectors for the events and event groups of interest. In someembodiments, the group vectors may be retrieved from a data store. Insome embodiments, the message vector may be regenerated from itsoriginal event, if it may be available. Also, in some embodiments, theclustering engine may be arranged to retrieve the message vector from adata store. In some embodiments, the original event may be stored andthe message vector may be associated with it as well. For example, inone or more of the various embodiments, both the original event and itsmessage vector may be stored together in a data store.

Accordingly, in some embodiments, for each event and event group pair,the clustering engine may be arranged to generate an outer product ofthe message vector and the group vector. In some embodiments, clusteringengines may be arranged to include hardware support for performingoperations such as generating outer produces from the message vector andgroup vector. In some embodiments, clustering engines may be arranged toemploy native features of CPUs or GPUs to generate the outer product.For brevity and clarity, the details of performing the outer productoperation are omitted here because one of ordinary skill in the art willappreciate that determining an outer product of two vectors is aconventional or well-known operation.

Accordingly, in some embodiments, if the user feedback indicates thatthe user intends to disassociate an event from an event group, theclustering engine may be arranged to subtract (component-wise) the outerproduct of the message vector and group vector from the matrix includedin the learner object.

In one or more of the various embodiments, if the user feedbackindicates that the user intends to associate an event with an eventgroup, the clustering engine may be arranged to add (component-wise) theouter product of the message vector and group vector to the matrixincluded in the learner object.

At block 1008, in one or more of the various embodiments, the clusteringengine may be arranged to employ the updated learner object to groupincoming events.

For example, in one or more of the various embodiments, clusteringengines may be arranged to generate agreement scores for a messagevector v and a group vector g by generating the outer product vg. Thisresult may be employed as a mask for the learner object matrix L bygenerating the Kronecker product between the two and retaining thoseentries of L for which there is a nonzero entry in vg (the outer productthe message vector and the group vector). Accordingly, the sum of thesenon-zero entries provides a scalar value that may be considered theagreement score.

In one or more of the various embodiments, if a learner object producesa large positive agreement score for an event and an event group, theclustering engine may be arranged to disregard a low similarity scoreand associate the event with the event group. Likewise, in one or moreof the various embodiments, if a learner object produces a largenegative agreement score for an event and an event group, the clusteringengine may be arranged to disregard a high similarity score and refrainfrom associating the event with the event group.

Next, in one or more of the various embodiments, control may be returnedto a calling process.

It will be understood that each block in each flowchart illustration,and combinations of blocks in each flowchart illustration, can beimplemented by computer program instructions. These program instructionsmay be provided to a processor to produce a machine, such that theinstructions, which execute on the processor, create means forimplementing the actions specified in each flowchart block or blocks.The computer program instructions may be executed by a processor tocause a series of operational steps to be performed by the processor toproduce a computer-implemented process such that the instructions, whichexecute on the processor, provide steps for implementing the actionsspecified in each flowchart block or blocks. The computer programinstructions may also cause at least some of the operational steps shownin the blocks of each flowchart to be performed in parallel. Moreover,some of the steps may also be performed across more than one processor,such as might arise in a multi-processor computer system. In addition,one or more blocks or combinations of blocks in each flowchartillustration may also be performed concurrently with other blocks orcombinations of blocks, or even in a different sequence than illustratedwithout departing from the scope or spirit of the invention.

Accordingly, each block in each flowchart illustration supportscombinations of means for performing the specified actions, combinationsof steps for performing the specified actions and program instructionmeans for performing the specified actions. It will also be understoodthat each block in each flowchart illustration, and combinations ofblocks in each flowchart illustration, can be implemented by specialpurpose hardware based systems, which perform the specified actions orsteps, or combinations of special purpose hardware and computerinstructions. The foregoing example should not be construed as limitingor exhaustive, but rather, an illustrative use case to show animplementation of one or more of the various embodiments of theinvention.

Further, in one or more embodiments (not shown in the figures), thelogic in the illustrative flowcharts may be executed using an embeddedlogic hardware device instead of a CPU, such as, an Application SpecificIntegrated Circuit (ASIC), Field Programmable Gate Array (FPGA),Programmable Array Logic (PAL), or the like, or combination thereof. Theembedded logic hardware device may directly execute its embedded logicto perform actions. In one or more embodiments, a microcontroller may bearranged to directly execute its own embedded logic to perform actionsand access its own internal memory and its own external Input and OutputInterfaces (e.g., hardware pins or wireless transceivers) to performactions, such as System On a Chip (SOC), or the like.

What is claimed as new and desired to be protected by Letters Patent ofthe United States is:
 1. A method for managing operations over a networkusing one or more network computers that include one or more processorsthat perform actions, comprising: providing an event that is associatedwith one or more operations in the network; employing a hash function togenerate one or more key values that correspond to one or more wordsincluded in message information that is associated with the event;generating a message vector based on the one or more key values, whereineach component in the message vector, that corresponds to a key value,is set to a value of one; determining one or more group vectors thathave a same number of components as the message vector, wherein eachgroup vector is associated with an event group; generating one or moresimilarity scores for the one or more group vectors based on the messagevector and the one or more group vectors, wherein each group vectorcorresponds to a separate similarity score; in response to a portion ofthe one or more similarity scores exceeding a threshold value,associating the event with one or more event groups, wherein each eventgroup is associated with a group vector that that corresponds to theseparate similarity score that exceeds the threshold value; andgenerating an interface that visually displays a plurality of eventgroups to a user, wherein in response to actions in the interface,performing actions, including: in response to the event being moved inthe interface from a first event group to a second event group, changingthe association of the event from the first event group to the secondevent group; and in response to the event being disassociated from anevent group in the interface, removing association of the event from allevent groups.
 2. The method of claim 1, further comprising: determininga learner object based on an association of the learner object with oneor more of a user, an account, a service, or an organization; employingthe learner object to generate one or more agreement scores based on themessage vector and the one or more group vectors, wherein each groupvector corresponds to a separate agreement score; in response to aportion of the one or more agreement scores exceeding an agreementthreshold value, associating the event with one or more event groups,wherein each event group is associated with the group vector thatcorresponds to the separate agreement score that exceeds the agreementthreshold value; and in response to another portion of the one or moreagreement scores being less than a disagreement threshold value,disassociating the event from each of the one or more event groups thatare associated with a group vector that is associated with the otherportion of agreement scores.
 3. The method of claim 1, furthercomprising: employing the hash function to generate one or moreadditional key values that correspond to one or more pairs of words thatare included in the message information; and including the one or moreadditional key values in the message vector, wherein each additional keyvalue corresponds to a component in the message vector that correspondsto another key value, is set to a value of one.
 4. The method of claim1, further comprising: determining non-semantic information from themessage information based on one or more of pattern matching, parsing,or regular expressions; and removing the non-semantic information fromthe message information, wherein the non-semantic information isexcluded from the message vector.
 5. The method of claim 1, whereinassociating the event with the one or more event groups, furthercomprises, adding the message vector to each group vector that isassociated with the one or more event groups.
 6. The method of claim 1,further comprising: providing feedback information from a user, whereinthe feedback is one or more or more of associating another event with anevent group or disassociating the other event with the event group;generating a learner object based on another message vector that isassociated with the other event and a group vector that is associatedwith the event group; and employing the learner object to generateagreement scores that are associated with the user.
 7. The method ofclaim 1, wherein generating the one or more similarity scores, furthercomprises, computing one or more cosine similarity values based on themessage vector and each of the one or more group vectors, wherein theone or more cosine similarity values are employed as the value of theone or more similarity scores.
 8. A system for managing operations fororganizations over a network, comprising: a network computer,comprising: a transceiver that communicates over the network; a memorythat stores at least instructions; and one or more processors thatexecute instructions that perform actions, including: providing an eventthat is associated with one or more operations in the network; employinga hash function to generate one or more key values that correspond toone or more words included in message information that is associatedwith the event; generating a message vector based on the one or more keyvalues, wherein each component in the message vector, that correspondsto a key value, is set to a value of one; determining one or more groupvectors that have a same number of components as the message vector,wherein each group vector is associated with an event group; generatingone or more similarity scores for the one or more group vectors based onthe message vector and the one or more group vectors, wherein each groupvector corresponds to a separate similarity score; in response to aportion of the one or more similarity scores exceeding a thresholdvalue, associating the event with one or more event groups, wherein eachevent group is associated with a group vector that that corresponds tothe separate similarity score that exceeds the threshold value; andgenerating an interface that visually displays a plurality of eventgroups to a user, wherein in response to actions in the interface,performing actions, including: in response to the event being moved inthe interface from a first event group to a second event group, changingthe association of the event from the first event group to the secondevent group; and in response to the event being disassociated from anevent group in the interface, removing association of the event from allevent groups and another network computer, comprising: anothertransceiver that communicates over the network; another memory thatstores at least instructions; and one or more processors that executeinstructions that perform actions, including: generating the event thatis provided to the network computer.
 9. The system of claim 8, whereinthe one or more network computer processors execute instructions thatperform actions, further comprising: determining a learner object basedon an association of the learner object with one or more of a user, anaccount, a service, or an organization; employing the learner object togenerate one or more agreement scores based on the message vector andthe one or more group vectors, wherein each group vector corresponds toa separate agreement score; in response to a portion of the one or moreagreement scores exceeding an agreement threshold value, associating theevent with one or more event groups, wherein each event group isassociated with the group vector that corresponds to the separateagreement score that exceeds the agreement threshold value; and inresponse to another portion of the one or more agreement scores beingless than a disagreement threshold value, disassociating the event fromeach of the one or more event groups that are associated with a groupvector that is associated with the other portion of agreement scores.10. The system of claim 8, wherein the one or more network computerprocessors execute instructions that perform actions, furthercomprising: employing the hash function to generate one or moreadditional key values that correspond to one or more pairs of words thatare included in the message information; and including the one or moreadditional key values in the message vector, wherein each additional keyvalue corresponds to a component in the message vector that correspondsto another key value, is set to a value of one.
 11. The system of claim8, wherein the one or more network computer processors executeinstructions that perform actions, further comprising: determiningnon-semantic information from the message information based on one ormore of pattern matching, parsing, or regular expressions; and removingthe non-semantic information from the message information, wherein thenon-semantic information is excluded from the message vector.
 12. Thesystem of claim 8, wherein associating the event with the one or moreevent groups, further comprises, adding the message vector to each groupvector that is associated with the one or more event groups.
 13. Thesystem of claim 8, wherein the one or more network computer processorsexecute instructions that perform actions, further comprising: providingfeedback information from a user, wherein the feedback is one or more ormore of associating another event with an event group or disassociatingthe other event with the event group; generating a learner object basedon another message vector that is associated with the other event and agroup vector that is associated with the event group; and employing thelearner object to generate agreement scores that are associated with theuser.
 14. The system of claim 8, wherein generating the one or moresimilarity scores, further comprises, computing one or more cosinesimilarity values based on the message vector and each of the one ormore group vectors, wherein the one or more cosine similarity values areemployed as the value of the one or more similarity scores.
 15. Aprocessor readable non-transitory storage media that includesinstructions for managing operations for organizations over a network,wherein execution of the instructions by one or more hardware processorsperforms actions, comprising: providing an event that is associated withone or more operations in the network; employing a hash function togenerate one or more key values that correspond to one or more wordsincluded in message information that is associated with the event;generating a message vector based on the one or more key values, whereineach component in the message vector, that corresponds to a key value,is set to a value of one; determining one or more group vectors thathave a same number of components as the message vector, wherein eachgroup vector is associated with an event group; generating one or moresimilarity scores for the one or more group vectors based on the messagevector and the one or more group vectors, wherein each group vectorcorresponds to a separate similarity score; in response to a portion ofthe one or more similarity scores exceeding a threshold value,associating the event with one or more event groups, wherein each eventgroup is associated with a group vector that that corresponds to theseparate similarity score that exceeds the threshold value; andgenerating an interface that visually displays a plurality of eventgroups to a user, wherein in response to actions in the interface,performing actions, including: in response to the event being moved inthe interface from a first event group to a second event group, changingthe association of the event from the first event group to the secondevent group; and in response to the event being disassociated from anevent group in the interface, removing association of the event from allevent groups.
 16. The media of claim 15, further comprising: determininga learner object based on an association of the learner object with oneor more of a user, an account, a service, or an organization; employingthe learner object to generate one or more agreement scores based on themessage vector and the one or more group vectors, wherein each groupvector corresponds to a separate agreement score; in response to aportion of the one or more agreement scores exceeding an agreementthreshold value, associating the event with one or more event groups,wherein each event group is associated with the group vector thatcorresponds to the separate agreement score that exceeds the agreementthreshold value; and in response to another portion of the one or moreagreement scores being less than a disagreement threshold value,disassociating the event from each of the one or more event groups thatare associated with a group vector that is associated with the otherportion of agreement scores.
 17. The media of claim 15, furthercomprising: employing the hash function to generate one or moreadditional key values that correspond to one or more pairs of words thatare included in the message information; and including the one or moreadditional key values in the message vector, wherein each additional keyvalue corresponds to a component in the message vector that correspondsto another key value, is set to a value of one.
 18. The media of claim15, further comprising: determining non-semantic information from themessage information based on one or more of pattern matching, parsing,or regular expressions; and removing the non-semantic information fromthe message information, wherein the non-semantic information isexcluded from the message vector.
 19. The media of claim 15, whereinassociating the event with the one or more event groups, furthercomprises, adding the message vector to each group vector that isassociated with the one or more event groups.
 20. The media of claim 15,further comprising: providing feedback information from a user, whereinthe feedback is one or more or more of associating another event with anevent group or disassociating the other event with the event group;generating a learner object based on another message vector that isassociated with the other event and a group vector that is associatedwith the event group; and employing the learner object to generateagreement scores that are associated with the user.
 21. The media ofclaim 15, wherein generating the one or more similarity scores, furthercomprises, computing one or more cosine similarity values based on themessage vector and each of the one or more group vectors, wherein theone or more cosine similarity values are employed as the value of theone or more similarity scores.
 22. A network computer for managingoperations for organizations over a network, comprising: a transceiverthat communicates over the network; a memory that stores at leastinstructions; and one or more processors that execute instructions thatperform actions, including: providing an event that is associated withone or more operations in the network; employing a hash function togenerate one or more key values that correspond to one or more wordsincluded in message information that is associated with the event;generating a message vector based on the one or more key values, whereineach component in the message vector, that corresponds to a key value,is set to a value of one; determining one or more group vectors thathave a same number of components as the message vector, wherein eachgroup vector is associated with an event group; generating one or moresimilarity scores for the one or more group vectors based on the messagevector and the one or more group vectors, wherein each group vectorcorresponds to a separate similarity score; in response to a portion ofthe one or more similarity scores exceeding a threshold value,associating the event with one or more event groups, wherein each eventgroup is associated with a group vector that that corresponds to theseparate similarity score that exceeds the threshold value; andgenerating an interface that visually displays a plurality of eventgroups to a user, wherein in response to actions in the interface,performing actions, including: in response to the event being moved inthe interface from a first event group to a second event group, changingthe association of the event from the first event group to the secondevent group; and in response to the event being disassociated from anevent group in the interface, removing association of the event from allevent groups.
 23. The network computer of claim 22, wherein the one ormore processors execute instructions that perform actions, furthercomprising: determining a learner object based on an association of thelearner object with one or more of a user, an account, a service, or anorganization; employing the learner object to generate one or moreagreement scores based on the message vector and the one or more groupvectors, wherein each group vector corresponds to a separate agreementscore; in response to a portion of the one or more agreement scoresexceeding an agreement threshold value, associating the event with oneor more event groups, wherein each event group is associated with thegroup vector that corresponds to the separate agreement score thatexceeds the agreement threshold value; and in response to anotherportion of the one or more agreement scores being less than adisagreement threshold value, disassociating the event from each of theone or more event groups that are associated with a group vector that isassociated with the other portion of agreement scores.
 24. The networkcomputer of claim 22, wherein the one or more processors executeinstructions that perform actions, further comprising: employing thehash function to generate one or more additional key values thatcorrespond to one or more pairs of words that are included in themessage information; and including the one or more additional key valuesin the message vector, wherein each additional key value corresponds toa component in the message vector that corresponds to another key value,is set to a value of one.
 25. The network computer of claim 22, whereinthe one or more processors execute instructions that perform actions,further comprising: determining non-semantic information from themessage information based on one or more of pattern matching, parsing,or regular expressions; and removing the non-semantic information fromthe message information, wherein the non-semantic information isexcluded from the message vector.
 26. The network computer of claim 22,wherein associating the event with the one or more event groups, furthercomprises, adding the message vector to each group vector that isassociated with the one or more event groups.
 27. The network computerof claim 22, wherein the one or more processors execute instructionsthat perform actions, further comprising: providing feedback informationfrom a user, wherein the feedback is one or more or more of associatinganother event and with an event group or disassociating the other eventwith the event group; and generating a learner object based on anothermessage vector the is associated with the other event and a group vectorthat is associated with the event group; and employing the learnerobject to generate agreement scores that are associated with the user.28. The network computer of claim 22, wherein generating the one or moresimilarity scores, further comprises, computing one or more cosinesimilarity values based on the message vector and each of the one ormore group vectors, wherein the one or more cosine similarity values areemployed as the value of the one or more similarity scores.